Fix flaky init_per_group in the MUC suite
#4241
Conversation
jacekwegr
changed the title
init_per_group in MUC suiteinit_per_group in the MUC suite
This comment was marked as outdated.
This comment was marked as outdated.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4241 +/- ##
==========================================
- Coverage 84.40% 84.39% -0.01%
==========================================
Files 552 552
Lines 33538 33538
==========================================
- Hits 28308 28305 -3
- Misses 5230 5233 +3 ☔ View full report in Codecov by Sentry. |
jacekwegr
force-pushed
the
muc-suite-fix-init-per-group
branch
from
7adbbef
to
ed2d95a
Compare
This comment was marked as outdated.
This comment was marked as outdated.
|
Do we know which suite left the state dirty with this helper still running? It'd be good to clean that up too 🤔 |
Yeah, it was |
There was a problem hiding this comment.
Nice that this is fixed. I am only thinking of a more generic solution, because this might appear again in a different place. Maybe we could amend http_helper:start/3 to have this check instead? If it's already running, it could log such information, then stop and start just like in this PR.
Otherwise, fixing the testcase that actually causes the problem might be the best option as @NelsonVides suggested.
jacekwegr
force-pushed
the
muc-suite-fix-init-per-group
branch
from
ed2d95a
to
a75eaf1
Compare
|
elasticsearch_and_cassandra_26 / elasticsearch_and_cassandra_mnesia / a75eaf1 small_tests_25 / small_tests / a75eaf1 small_tests_26 / small_tests / a75eaf1 small_tests_26_arm64 / small_tests / a75eaf1 ldap_mnesia_25 / ldap_mnesia / a75eaf1 dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / a75eaf1 dynamic_domains_mysql_redis_26 / mysql_redis / a75eaf1 ldap_mnesia_26 / ldap_mnesia / a75eaf1 internal_mnesia_26 / internal_mnesia / a75eaf1 dynamic_domains_pgsql_mnesia_26 / pgsql_mnesia / a75eaf1 pgsql_mnesia_25 / pgsql_mnesia / a75eaf1 pgsql_cets_26 / pgsql_cets / a75eaf1 pgsql_mnesia_26 / pgsql_mnesia / a75eaf1 mysql_redis_26 / mysql_redis / a75eaf1 mssql_mnesia_26 / odbc_mssql_mnesia / a75eaf1 muc_SUITE:register:user_submits_registration_form{error,
{{assertion_failed,assert,is_iq_result,
[{xmlel,<<"iq">>,
[{<<"type">>,<<"set">>},
{<<"id">>,<<"328b69ef1f57d12be46556e3e578fdf5">>},
{<<"to">>,<<"muc.localhost">>}],
[{xmlel,<<"query">>,
[{<<"xmlns">>,<<"jabber:iq:register">>}],
[{xmlel,<<"x">>,
[{<<"xmlns">>,<<"jabber:x:data">>},
{<<"type">>,<<"submit">>}],
[{xmlel,<<"field">>,
[{<<"type">>,<<"hidden">>},
{<<"var">>,<<"FORM_TYPE">>}],
[{xmlel,<<"value">>,[],
[{xmlcdata,<<"jabber:iq:register">>}]}]},
{xmlel,<<"field">>,
[{<<"type">>,<<"text-single">>},
{<<"var">>,<<"nick">>}],
[{xmlel,<<"value">>,[],
[{xmlcdata,
<<"thirdwitchroom-965aabd12c">>}]}]}]}]}]}],
{xmlel,<<"iq">>,
[{<<"from">>,<<"muc.localhost">>},
{<<"to">>,
<<"alice_user_submits_registration_form_2886@localhost/res1">>},
{<<"type">>,<<"error">>},
{<<"id">>,<<"328b69ef1f57d12be46556e3e578fdf5">>}],
[{xmlel,<<"query">>,
[{<<"xmlns">>,<<"jabber:iq:register">>}],
[{xmlel,<<"x">>,
[{<<"xmlns">>,<<"jabber:x:data">>},
{<<"type">>,<<"submit">>}],
[{xmlel,<<"field">>,
[{<<"type">>,<<"hidden">>},
{<<"var">>,<<"FORM_TYPE">>}],
[{xmlel,<<"value">>,[],
...dynamic_domains_mssql_mnesia_26 / odbc_mssql_mnesia / a75eaf1 mssql_mnesia_26 / odbc_mssql_mnesia / a75eaf1 graphql_muc_light_SUITE:admin_cli:admin_muc_light:admin_create_room_with_unprepped_id{error,
{#{what => invalid_response_code,expected_type => ok,
response_code => {exit_status,1}},
[{graphql_helper,assert_response_code,2,
[{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
{line,256}]},
{graphql_helper,get_ok_value,2,
[{file,"/home/circleci/project/big_tests/tests/graphql_helper.erl"},
{line,239}]},
{graphql_muc_light_SUITE,admin_create_room_with_unprepped_id,1,
[{file,
"/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE.erl"},
{line,1156}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1302}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1234}]}]}}graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:admin_create_room_with_unprepped_id{error,
{{badmatch,null},
[{graphql_muc_light_SUITE,admin_create_room_with_unprepped_id,1,
[{file,
"/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE.erl"},
{line,1155}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1302}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1234}]}]}}graphql_muc_light_SUITE:domain_admin:domain_admin_muc_light:end_per_group{error,
{{unregistering_failed,
{amount,1},
{unregistered_items,
[{{<<"_domain_admin_invite_user_no_permission_1111">>,
[{escalus_event_mgr,<0.25122.0>},
{tc_name,domain_admin_invite_user_no_permission},
{escalus_cleaner,<0.25121.0>},
{watchdog,<0.25120.0>},
{muc_light_host,<<"muclight.localhost">>},
{secondary_muc_light_host,<<"muclight.localhost.bis">>},
{protocol,http},
{domain_admin,{<<"admin@localhost">>,<<"3714b64ac7a829f2">>}},
{schema_endpoint,domain_admin},
{{ejabberd_cwd,mongooseim@localhost},
"/home/circleci/project/_build/mim1/rel/mongooseim"},
{preset,"odbc_mssql_mnesia"},
{mim_data_dir,
"/home/circleci/project/big_tests/tests/graphql_muc_light_SUITE_data"},
{tc_logfile,
"https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4241/212028/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%4065a864254808.2024-03-13_08.44.46/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-13_08.49.13/graphql_muc_light_suite.domain_admin_invite_user_no_permission.html"},
{tc_group_properties,[{name,domain_admin_muc_light}]},
{tc_group_path,[[{name,domain_admin}]]},
{data_dir,
"/home/circleci/project/big_tests/_build/default/lib/mongoose_tests/ebin/graphql_muc_light_SUITE_data/"},
{priv_dir,
"https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/4241/212028/odbc_mssql_mnesia.26.1.2-amd64/big/ct_run.test%4065a864254808.2024-03-13_08.44.46/big_tests.tests.graphql_muc_light_SUITE.logs/run.2024-03-13_08.49.13/log_private/"},
{{saved_modules,mongooseim@localhost,<<"localhost">>},
#{mod_vcard =...mssql_mnesia_26 / odbc_mssql_mnesia / a75eaf1 |
There was a problem hiding this comment.
It's ok for me. @NelsonVides do you think we need to fix the root cause in this PR, or shall we merge this?
This PR fixes a problem where tests might fail if an HTTP listener has already been started in the
init_per_groupfunction of the MUC suite.Sometimes, if other tests that need an HTTP listener don't work right, they can leave the listener running. This causes a problem when trying to start it again in the
init_per_group.